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RELATED APPLICATION 
This application claims the benefit of Provisional Application, U.S. 
Serial No. 60/184,088, filed on February 22, 2000, entitled "SUPER JPEG BUILDER 
AND METHOD FOR IMAGE ENHANCEMENT", by Kenneth Richards, Thomas 
Ramsay, and Michael Wackerfuss. 

TECHNICAL FIELD 
The present invention relates to image processing, more particularly, to 
an image processing system and method for automatically processing images without 
operator's intervention. 

BACKGROUND OF THE INVENTION 
Considerable research and development have been conducted over the 
past few years in image processing fields, for example, image compression to reduce an 
image file size, or image enhancement to improve an image quality, etc. 

Typically, image data are large in size and often required to be 
compressed to a smaller size before they are stored and/or transmitted in media. 
Various methods have been developed to compress an image file. For example, in U.S. 
Patent No. 5,426,512, a method of digital compression of images is disclosed. The 
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patented method uses a DCT transform coefficients yielded from a Discrete Cosine 
Transform (DCT) of all the blocks as well as other display and perceptual parameters to 
generate a quantization matrix which in turn yields a reproduced image having a low 
perceptual error. The patented method adapts or customizes the individual quantization 
5 matrix to the image being compressed. 

One of the problems associated with existing image compression 
methods is that these compression methods do not apply any pre-compression filtering 
to reduce image quality degradation as the compression is applied, e.g. sharpening of 
images, removing artifacts, etc. Under a high compression setting, an image generally 

10 loses its image characteristics/attributes. Further, compression of images introduces 
artifacts. These methods of compression do not take into account for preparing the 
image prior to compression to reduce the visible artifacts so as to maximize the 
compression applied and retain visual qualities. Images that have been 1) previously 
compressed with an algorithm, for example, JPEG (Joint Photographic Expert Group); 

15 2) having a dot pattern from being scanned; 3) GIF images that have dots because of 
their dithering or color pallet being limited to some number of colors, have varying 
degrees of artifacts dependent upon the compression applied. 

Some existing image processing systems apply enhancement to original 
and previously compressed images to enhance the image quality. When automated, the 

20 same enhancement parameters are applied to all images of the batch process such as 
contrast or brightness without taking into account for the individual qualities of each 
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image to be enhanced. Thus, these existing image enhancement systems may not 
enhance a batch processed image, but degrade the quality and/or visual characteristics of 
the image as viewed by a human. In addition, artifacts problems still remain. 

Another problem that resides in conventional image enhancement 
5 systems is that the image processing requires an operator's intervention. Typically, an 
operator randomly chooses image processing parameters, such as sharpness, contrast 
percentage, etc., enhances an image based on the chosen parameters, and judges whether 
the enhanced image is acceptable. If not, different image processing parameters are 
chosen to enhance an original image. This process is time consuming. This process is 

10 more or less depending on operator's experience. U.S. Patent No. 5,694,484 has 

addressed these concerns and attempts to solve the problem. However, the system still 
requires an operator's intervention, even though it is minimized. For example, an 
operator needs to select image processing operations to process an input image. 
Further, as mentioned above, like many other existing image enhancement systems, this 

1 5 patented system does not enhance an image to a point that the image has quality and/or 
visual characteristics as viewed by a human and does not pre-process the image prior to 
compression so as to reduce image quality degradation or artifacts introduced by the 
compression method and thus allow a greater degree of compression to be applied while 
retaining visual qualities. 

20 Furthermore, existing enhancement and compression methods do not 

take into account for image quality such as sharpness as the size of the image is reduced. 
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This is particularly true with thumbnail images where the amount of information that 
defines the image and its characteristics are very small. Thus, small images, such as 
thumbnails, have poor visual quality. 

A further problem of conventional image enhancement systems is that an 
input image must be loaded into a memory repeatedly for each multiple output desired. 
This slows down the processing time and wastes many resources. 

It is with respect to these and other considerations that the present 
invention has been made. 
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SUMMARY OF THE INVENTION 
In accordance with this invention, the above and other problems were 
solved by providing an image processing system and method capable of automatically 
analyzing, classifying, enhancing, pre-compression processing, and compressing each of 
5 a series of input images on-the-fly without operator's intervention in a wire and/or 
wireless transmission environment or embedded in a hardware, software, or firmware 
device, and outputting one or more enhanced images without re-analyzing, re- 
classifying, and re-enhancing the input image. 

The present invention provides an image processing system and method 
10 which automatically generates one or more highest quality images in given file sizes. 
Also, the present invention provides a system capable of processing a series of input 
images in a batch mode or in real time on-the-fly. Accordingly, the system in 
accordance with the principles of the present invention can function as a round-the- 
clock "professional image processing specialist" with speed and skill levels unmatched 
15 by any human counterpart. 

Further, the present invention provides a system and method which 
greatly improve high-contrast or over- or under-exposed pictures, images generated 
from document scanners or digital cameras, etc. 

Furthermore, the present invention provides an image processing system 
20 and method for generating enhanced and/or compressed images having a high 

compression to quality ratio, such as a JPEG image, and enhanced uncompressed 
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images, such as a BMP (Bitmap) or TIFF (.tif) image. For example, the system 
produces an image having quality and visual characteristics as viewed by a human 
including image brightness, contrast, and sharpness as compared with an original 
uncompressed image, while reducing the amount of memory required for storing the 
5 file. 

In one embodiment of the present invention, each input image is 
analyzed for its unique luminosity and/or color characteristics/attributes, classified, and 
enhanced to improve its visual qualities. The method includes automatically analyzing 
a luminosity histogram associated with a particular image and deriving at least three 

10 statistical functions for the luminosity histogram, the statistical functions include a 

Weighted Mean (WM) function, a Half Value Position (HVP) function, and a Weighted 
Standard Deviation (WSD) function; and automatically classifying the image by its 
statistical functions, which are compared against set parameters for each class of images 
that are derived from experimental data stored in tables or a database structure. The 

1 5 experimental data can be a collection of human experienced/perceptive data about 

images from which tables or a knowledge based database is built. The method further 
includes automatically applying a predetermined set of changes to the image based upon 
its class. 

The system and method in accordance with the present invention further 
20 include all interfaces and logics capable of operating in real time and static 

environments as part of an application or as a stand-alone utility, for example, on a 



6 

Attorney Docket No. 13095.1-US-U1 



stand-alone computer, a server-based network, or the Internet, etc. The interfaces 
provide control of an operator defined quality and compression settings which can be 
defined on an individual image basis or specified as a rule set for all images processed 
until such time as another rule set is applied. These settings define the methodology for 
5 analysis of the file type to achieve the compression to quality ratio specified. In 
addition, the system and method are capable of performing pre-image compression 
enhancement for any known or future compression algorithm, such as JPEG, Wavelet, 
Fractal, etc. This is also made possible for outputting an enhanced uncompressed image 
file type, such as BMP or TIFF, etc. 

10 The present invention also addresses the issues of artifacts introduced 

into images due to the source or previous processing of the image. The artifacts are 
reduced by applying filters, such as a "Blur" filter that tends to "smear/smudge" the 
blocking generated as a result of compression or similar to a blocking of dot patterns 
created by scanned images or GIF. The "Blur" filter reduces the appearance of the 

1 5 blocking throughout the whole picture. The blocking artifacts and the picture details are 
both blurred. However, the edges that define the original picture content can be restored 
by applying multiple iterations of the unsharp mask filter in varying amounts (e.g. 
15%/15%/15%) whereupon the edges are slowly redefined. Furthermore, the pixel size 
of the image may determine the amount of unsharp mask applied. 

20 Furthermore, in an automated environment where diverse image sizes 

and quality need to be enhanced and/or compressed or recompressed, a ratio of file size 
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in bytes to the total pixel count of an image can be used as a measuring parameter to 
determine an optimal compression factor so as to retain good visual qualities and 
achieve a higher degree of compression than that of an original compressed image, e.g. 
a JPEG. This allows an image processing system to approximate a quality based upon 
5 the image's ratio. 

In one embodiment, the present invention provides a method of 
automatically processing a series of input images without operator's intervention. In 
automatically processing each of the input images, the method includes the steps of 
providing processing parameters; automatically analyzing, classifying, and enhancing 
1 0 each of the input images without operator's intervention; and outputting at least one 
enhanced image. 

Still in one embodiment, the method outputs more than one enhanced 
images without re-analyzing, re-classifying, and re-enhancing the input image. The 
enhanced output images may be in different file types and/or sizes. 

1 5 Further in one embodiment, the images can be processed in real time on- 

the-fly or in a batch mode. Further, the images can be processed in a hardware, 
firmware, or software embodiment that furthermore can be embedded in another system, 
device, and/or application. 

Yet in one embodiment, the method may include one or more of the 

20 steps of: pre-processing the input image; automatically resizing the enhanced image; 
automatically pre-compression enhancing the enhanced image; post-processing the 
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enhanced image; and automatically compressing the enhanced image. 

The present invention also provides a method of automatically 
processing a series of input images without operator's intervention. In automatically 
processing each of the input images, the method includes the steps of providing 
processing parameters; automatically analyzing and classifying the input image without 
operator's intervention; automatically compressing the image; and outputting at least 
one compressed image. 

Still in one embodiment, the method outputs more than one compressed 
image without re-analyzing and re-classifying the input image. The compressed output 
images may be in different file types and/or sizes, and/or of different compression 
ratios, YUV settings, and Q-factor. 

Further in one embodiment, the images can be processed in real time on- 
the-fly or in a batch mode. Further, the images can be processed in an embedded 
hardware, firmware, or software that can be embedded in another application. 

Yet in one embodiment, the method may include one or more of the 
steps of: pre-processing the input image; resizing the compressed image; pre- 
compression enhancing the compressed image, post-processing the compressed image; 
and compressing the compressed image. 

The present invention further provides an image processing system of 
automatically processing a series of input images without operator's intervention. In 
one embodiment, the system includes an input module for inputting a set of parameters; 
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an image analysis module for automatically analyzing each of the input images, an 
image classification module for automatically classifying the analyzed image, an image 
enhancement module for automatically enhancing the classified image based on 
classification, and an output module for outputting at least one enhanced image. 
5 Still in one embodiment, the output module outputs enhanced images 

with different file types and/or sizes without re-analyzing, re-classifying, and re- 
enhancing the input image. 

Further in one embodiment, the image processing system processes the 
images in real time on-the-fly or in a batch mode. 

10 Additionally in one embodiment, the image processing system can be 

embedded in a hardware, firmware, or software which can be embedded in another 
system or application. 

Yet in one embodiment, the system may include one or more of the 
following: a pre-processor for pre-processing the input image; a resizer for resizing the 

1 5 enhanced image; a pre-compression enhancement module for pre-compression 
enhancing the enhanced image; a post-processor for post-processing the enhanced 
image; and a compression module for automatically compressing the enhanced image. 

Additionally in one embodiment, the image analysis module analyzes the 
image based on an image histogram. 

20 Still in one embodiment, the image classification module classifies the 

image based on the Weighted Means range, the Weighted Standard Deviation range, 
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and the Halve Value Position range of an image luminosity histogram. 

Further in one embodiment, the image enhancement module may include 
a brightness filter, a contrast filter; a saturation filter; a curve filter; an unsharp mask 
filter, a blur filter, and/or a hue filter, etc. 
5 Yet in one embodiment, the image compression module includes a JPEG 

compression. 

Still in one embodiment, the image resizer resizes the original input 
image to any smaller size image proportionately to the original image using any 
specified axis. 

10 Further in one embodiment, the image resizer resizes the image to a 

thumbnail size. 

Additionally in one embodiment, the image pre-processor includes a 
brightness filter, a contrast filter, a saturation filter, and/or an unsharp mask filter. 

Further in one embodiment, the image post-processor includes a 
1 5 brightness filter, a contrast filter, a saturation filter, and/or an unsharp mask filter. 

One aspect of the present invention is that the automatic image 
processing operations can be manipulated by an operator. For example, an operator can 
interrupt the automatic image processing operations at any time. The images in the 
process can be saved in a memory and later restored to continue the operations. For 
20 example, if an operator wants to check the quality of the process and/or the quality of a 
specific image, the images being processed in the operations are saved automatically. 
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A further aspect of the present invention is that the system and method 
provide an interactive interface between the system and an operator. For example, the 
present invention may generate an enhanced image on-the-fly or in real time to allow an 
operator or user to preview the enhanced image before the system outputs the enhanced 
5 image in a given size. 

For a better understanding of the invention, reference should be made to 
the drawings, which form a further part hereof, and to accompanying descriptive matter 
in which there are illustrated and described specific examples in accordance with the 
invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Referring now to the drawings in which like reference numbers represent 
corresponding parts throughout: 

Fig. 1 illustrates one embodiment of an image processing system in 
accordance with the principles of the present invention. 

Fig. 2 illustrates one embodiment of an image processing system having 
additional optional modules in accordance with the principles of the present invention. 

Fig. 3 illustrates another embodiment of an image processing system in 
accordance with the principles of the present invention. 

Fig. 4 illustrates another embodiment of an image processing system 
having additional optional modules in accordance with the principles of the present 
invention. 

Figs. 5A-5B illustrate a flow chart diagram of one embodiment of an 
image processing method in accordance with the principles of the present invention. 

Figs. 6A-6C illustrate a flow chart diagram of image 
classification/enhancement operations in one embodiment of an image processing 
method in accordance with the principles of the present invention. 

Figs. 7A-7C are exemplary curves applied to an image in an image 
enhancement operation in accordance with the principles of the present invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



In the following description of the illustrated embodiments, reference is 
made to the accompanying drawings which form a part hereof, and in which is shown 
by way of illustration several embodiments in which the invention may be practiced. It 
5 is to be understood that other embodiments may be utilized as structural or operational 
changes and may be made without departing from the spirit and scope of the present 
invention. 

Fig. 1 illustrates one embodiment of an image processing system 100 in 
accordance with the principles of the present invention. The image processing system 

10 100 includes an image analysis module 102, an image classification module 104, and 
an image enhancement module 106 wherein each image is automatically analyzed, 
classified, and enhanced individually based upon each of the characteristics without 
human intervention when multiple images are submitted for processing. It is noted that 
the image processing system 100 is capable of processing a single image as well as a 

15 series of images. 

As shown in Fig. 1 , an image or a series of images are input into the 
image processing system 100. A set of processing parameters are used in the image 
processing system 100. The processing parameters may include predefined system, 
device, or operating environment parameters. The predefined system, device, or 

20 operating environment parameters may be determined by system resources and 

transmission bandwidth available or applications that the system is transmitted to or 
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embedded therein, etc. The processing parameters may also include user input 
parameters. For example, the processing parameters may include the number of output 
images, the size of the output image that a user desires, and/or a set of image output 
quality and compression settings, etc. The settings can be made on an individual image 
5 basis or specified in a rule set for all images processed until such time as another rule 
set is applied. The settings may define the methodology for analysis of a file type to 
achieve the image compression to quality ratio desired. Further, if a user chooses not to 
define its own settings, a set of default settings may be used. 

The input images may be processed in the image processing system 100 
10 in real time on-the-fly or in a batch mode. The image processing system 1 00 in 

accordance with the present invention may also be applied to environments such as 
web-crawling, a program that searches web pages hosted on the Internet to extract 
images for purpose of enhancing and further compressing of the images in an automated 
environment. 

1 5 Additionally in one embodiment, the image processing system can be 

embedded in a hardware, firmware, or software which can be embedded in another 

system or application. 

Further, the system 100 may be applied in an automated on-the-fly 

environment for re-purposing and further compressing of images. The re-purposing and 
20 further compressing of image allows for transmission of images to diverse devices 

having different rendering capacity and image type requirements and further reduction 
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of the bandwidth required for transmission of the images and the time required to 
download the images. The process may be applied to fixed wire or wireless 
transmission of images to devices. One example is that the system applies to images 
hosted on the Internet where an image content of an HTML page is further compressed 
5 and possibly re-purposed as a black and white image to fit the rendering requirements of 
a device. 

Further, the system may be applied in an automated on-the-fly 
environment where images are sent to an imaging server via the Internet whereupon the 
images are processed directly upon receipt or from a temporary storage, stored for 

10 retrieval, and/or transmitted to the point of origin or some other defined location. In 
addition, the system may be applied in an automated on-the-fly or static environment 
where an image of a rendered HTML page is captured, enhanced, compressed, and/or 
stored and/or transmitted. 

In Fig. 1, the image analysis module 102 analyzes a luminosity 

1 5 histogram associated with a particular image and derives the Weighted Mean (WM), 
Half Value Position (HVP), and Weighted Standard Deviation (WSD) of the image 
luminosity histogram. The analyzed image is classified by the image classification 
module 104 and enhanced by the image enhancement module 106 based on the 
classification. 

20 In Fig. 2, the image processing system 100 may include an image input 

module 108, a pre-processor 1 10, a resizer 1 12, a pre-compression enhancement module 
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1 14, a post-processor 1 16, a compression module 118, and an output module 120. For 
example, the resizer 112 may resize an enhanced image to a file size defined by a user, 
for example, to a thumbnail size (e.g. the longest axis of an image having less than 128 
pixels). In another example, the image compression module 118 may include a JPEG 
5 compression algorithm to compress an image if desired by the user. It is appreciated 
that these additional modules may be optional, and that the system 100 is not limited to 
these optional modules. 

The pre-processor 110 and the post-processor 116 may include a set of 
filters, but not limited to, for adjusting the brightness, contrast, and/or saturation, or 

1 0 applying unsharp mask to the image to enhance the image quality of the processed 

image. It is noted that the terms for brightness, contrast, saturation, and unsharp mask 
are known in the image processing art. 

The input module 108 sets or preserves a white threshold for an input 
image and determines whether an input image file has a file extension of jpg, .bmp, or 

15 .tif, etc. The input module 108 may also load the input image into a memory in a 

standard format, for example, a Device Independent Bitmap (DIB) format. The input 
module 108 determines image attributes, such as file type, file size, file dimension, etc., 
and obtains an image histogram, such as an image luminosity histogram to calculate the 
Weighted Mean (WM), Half Value Position (HVP), and Weighted Standard Deviation 

20 (WSD) of an image. 

The output module 120 may save the output image to a storage medium, 
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transmit the output image to a local or remote device, or pass the output image to 
another application, etc. If more than one images are desired to be outputted-for 
example, a user desires to have a first enhanced image to be of a regular size and a 
second enhanced image to be of a thumbnail size--, the system 100 automatically 
5 generates the second output image based on the user defined file size after generating 
the first output image. The system 100 does not need to re-load into memory, re- 
analyze, re-classify, or re-enhance the original input image. The system 100 processes 
the additional output images based on the user defined parameters for the additional 
output images for example resizing, pre-compression enhancement, pre- and post- 

10 processing, output file type, and compression after the enhancement module 106. 

Further, since the image input, the image analysis, image classification, 
image enhancement, and the image output can be separable, the system 100 is able to 
automatically process the subsequent input image in the image analysis module 102, 
image classification module 104, and image enhancement module 106 while outputting 

1 5 previously enhanced images. 

In one embodiment of the present invention, the image analysis module 
102 analyzes the input image to determine its unique characteristics, for example, color, 
black/white (B/W), gray scale, luminosity, contrast, brightness, curve, etc. The 
analyzed data may be stored in tables, a knowledge based database, or the like. 

20 In one embodiment of the present invention, the image classification 

module 104 classifies the input image based upon the Weighted Mean (WM), Half 
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Value Position (HVP), and Weighted Standard Deviation (WSD) of the image 
luminosity histogram. The WM, HVP, and WSD are compared to a predetermined set 
of parameters for each class of images that are derived from experimental data stored in 
tables, a knowledge based database, or the like. The experimental data can be a 
5 collection of human experienced/perceptive data about images from which tables or a 
knowledge based is built. Then, a predetermined set of changes may be applied to the 
image based upon its class. 

In one embodiment of the present invention, the image enhancement 
module 106 provides filters and masks to change the characteristics of the input image 

10 such that the image is improved and enhanced for its visual qualities. 

In one embodiment of the present invention, the image processing 
system 1 00 provides a user interface which allows a user to define an end-product 
image characteristics, for example, the highest compression, the best compression to 
quality ratio, a specific or near loss-less image characteristic, etc. 

15 Fig. 3 illustrates another embodiment of the image processing system 

122 in accordance with the principles of the present invention. The system 122 includes 
the analysis module 102, the classification module 104, and a compression module 124. 
The image is compressed by the compression module 124. 

In Fig. 4, the system 122 may further include the image input module 

20 108, the pre-processor 110, the resizer 1 12, the pre-compression enhancement module 
1 14, the post-processor 116, the compression module 118, and the output module 120. 
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It is appreciated that these additional modules may be optional, and that the system 122 
is not limited to these optional modules. 

Figs. 5A-5B illustrate a flow chart diagram of one embodiment of an 
image processing method in accordance with the principles of the present invention. 
5 As shown in Fig. 5 A, an image processing operation flow 126 starts with 

an operation of setting a preserve white threshold for an input image in an operation 
128. In one embodiment, the preserve white threshold may be set to 253-255 out of a 
gray scale of 0-255. Accordingly, for the portions which have the gray scale of 253 or 
more, the white is preserved in the subsequent image processing. It is appreciated that a 

10 border can be adjusted and/or preserved with a certain gray scale within the scope of the 
present invention. Then, the operation flow 126 determines whether the input image is 
a compressed image, such as a JPEG image or an uncompressed image, such as a BMP 
or TIFF, in an operation 130. In one implementation shown in Fig. 5 A, if the input 
image is not a JPEG, BMP, or TIFF file, the operation flow 126 may be terminated. It 

15 is appreciated that the implementation can be varied such that other types of compressed 
or uncompressed images can be processed without departing from the scope of the 
present invention. 

In Fig. 5A, if the input image is a JPEG, BMP, or TIFF, the image is 
loaded into a memory or buffer in a standard format, such as a DIB format, in an 

20 operation 132. Then, the image attributes, such as file type, file size, and file 

dimension, are determined in an operation 134 by determining the characteristics of a 



20 

Attorney Docket No. 13095. 1-US-U1 



physical file. It is appreciated that the file type, file size, and file dimension 
determination is known in the art and not the focus of the present invention. Next, an 
image luminosity histogram of the input image is generated in an operation 136. The 
image luminosity histogram may include Weighted Mean (WM), Weighted Standard 
5 Deviation (WSD), and Half Value Position (HVP). The details of the WM, WSD, and 
HVP are described later in details. It is appreciated that the other types of image 
histograms can be used without departing from the scope of the present invention. 

Still in Fig. 5 A, the operation flow 126 determines whether a user desires 
to pre-process the input image in an operation 138. If "Yes", the brightness, contrast, 

1 0 and saturation percentage of the image may be adjusted based on the user definition in 
operations 140, 142, and 144, respectively. Further, an unsharp mask may be applied to 
the image, for example applying a mask percentage of a radius equal to 1 and a 
threshold equal to 1 in an operation 146. The pre-processing mask percentage can also 
be defined by the user. It is appreciated that the sequence or order of the operations 

15 140-146 can be varied within the scope of the present invention. 

If the image pre-processing is not desired in the operation 138, i.e. "No", 
the image is analyzed/classified based on the image histogram statistics in an operation 
148. The detailed analysis/classification is described later in Figs. 6A-6C. 

Then, the operation flow 126 determines whether an image enhancement 

20 is desired in an operation 150. If "Yes", a curve is applied based on analysis of the 
image histogram statistics in an operation 152. Further, the percentage of brightness, 
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contrast, and saturation of the image is adjusted based on the analysis of the image 
histogram statistics in operations 154, 156, and 158, respectively. It is appreciated that 
the sequence or order of the operations 152-158 can be varied within the scope of the 
present invention. Then, the operation flow 126 goes to Fig. 5B. On the other hand, if 
5 the image enhancement is not desired in the operation 150, i.e. "No", then the operations 
152-158 are bypassed, and the operation flow 126 goes to Fig. 5B directly. 

In Fig. 5B, the operation flow 126 determines whether the user desires to 
resize the image in an operation 160. If "Yes", the image is resized along its X axis, Y 
axis, or the longest axis in an operation 162. Then, the operation flow 126 determines 

10 whether the resized image is a thumbnail size image (e.g. the longest axis of the image 
having less than 128 pixels) in an operation 164. If the resized image is a thumbnail 
image, i.e. "Yes", the brightness of the image is adjusted by a predefined percentage, 
such as 2%, etc. in an operation 166, and an unsharp mask is applied by a predetermined 
percentage having a predetermined radius and/or threshold, for example, by 1 0% having 

15 a radius of 1 and threshold of 0 in an operation 168. 

If the resized image is not a thumbnail size image in the operation 164, 
i.e. "No", or if the image is not desired to be resized in the operation 160, i.e. "No", then 
the operation flow 126 determines whether an unsharp mask should be applied based on 
analysis of the image histogram statistics in an operation 170. If an unsharp mask 

20 should be applied, i.e." Yes", the operation flow 126 determines whether the input image 
is an uncompressed image, such as a BMP or TIFF image, or the input image is a 
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compressed image, such as a JPEG image in operations 172, 174. If the input image is a 
BMP or TIFF image, then the operation flow 126 determines whether the largest 
dimension of the input image is between a range, for example, between but not 
including 128 and 450 pixels in an operation 176. If the largest dimension of the input 
5 image is within 128-450 pixel range, i.e. "Yes", an unsharp mask is applied by a 

predetermined percentage in an operation 178, for example, by 30%, which has a radius 
and threshold based on the image analysis. Next, the operation flow 126 goes to 
determine whether an image post-processing is desired in an operation 182. 

If the largest dimension of the input image is not within 128-450 pixel 
10 range in the operation 176, i.e. "No", or if the input image is a JPEG, BMP, or TIFF 

image from the operations 172 and 174, then an unsharp mask is applied in an operation 
1 80 by a percentage, radius, and threshold based on the image analysis. Next, the 
operation flow 126 goes to determine whether an image post-processing is desired in an 
operation 182. 

15 If the user desires to post-process the image in the operation 1 82, i.e. 

"Yes", the brightness, contrast, and saturation percentage of the image may be adjusted 
by the user's definition in operations 184, 186, and 188, respectively. Further, an 
unsharp mask may be applied to the image by post-processing mask percentage, for 
example, having a radius equal to 1 and a threshold equal to 1 in an operation 190. The 

20 post-processing mask percentage can also be defined by the user. It is appreciated that 
the sequence or order of the operations 184-190 can be varied within the scope of the 



23 

Attorney Docket No. 13095. 1-US-U1 



present invention. 

If the post-processing is not desired in the operation 182, i.e. "No", then 
the operation flow 126 may determine whether a trial image or a preview of an output 
image is desired in an operation 192. If a trial image is desired, i.e. "Yes", then a logo, 

5 such as a company's logo, e*g. VisualGold™ logo, may be embossed in the trial image 
in an operation 194. Next, the operation flow 126 goes to determine whether an image 
output is a JPEG image in an operation 196. 

If the trial image is not desired in the operation 194, then the operation 
flow 126 goes to determine whether an image output is a JPEG image in the operation 

10 196. Also, since each of the processing steps is separable from each other, the operation 
flow 126 can be interrupted at any time. Once interrupted, the image at the time of 
interruption is automatically saved for review and further processing. 

If the image output is a JPEG image in the operation 196, i.e. "Yes", then 
the YUV color settings and Q-factor for the JPEG image are set in an operation 198. 

1 5 The YUV color settings and Q-factor are described later in details. Next, the image 

output is saved to a storage medium, transmitted to a device, or passed to an application, 
etc., in an operation 200. If the image output is not a JPEG image in the operation 196, 
then the operation 198 is bypassed, and the operation flow 126 goes to an operation 200 
to save the image output to a storage medium, or to transmit the image output to a 

20 device, or to pass the image output to another application, etc. 

When a compression is desired to drive an output file to a specific file 
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size, the operation flow 126 further includes an operation 201 to determine YUV 
settings and Q-factor. 

Then, the operation flow 126 goes to determine whether additional image 
outputs of the same input image are desired in an operation 202. If additional image 
5 outputs are not desired, the image processing operation flow 126 is terminated. If 
additional image outputs are desired, the image processing operation flow 126 goes 
back to the operation 160 to determine whether the additional image needs to be resized 
and continues the steps as described above. It is noted that the additional output images 
are generated without re-analyzing, re-classifying, or re-enhancing/re-compressing the 
10 input image. 

It is appreciated that additional operations can be implemented within the 
scope of the present invention. For example, the operation flow 126 may include 
operations such that the processed image can be sampled and compared to the input 
image to allow a user to adjust the processing parameters to achieve the desired output 

1 5 images. Accordingly, the image processing method may include steps of establishing 
the need for further processing, and providing successive filter settings for each 
consecutive filter pass by a measurable parameter or a change between input image and 
processed image, and its approximation to a default or user defined parameter. It is 
noted that a database containing all process information unique to the image 

20 characteristics upon which the process draws information for further processing can be 
provided. Further, it is noted that an archive of pre-processing and post-processing 
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settings for all images previously processed can be provided. All images previously are 
processed for a given set of user defined characteristics upon which the process draws 
information for all future image processing to reduce process iterations for desired 
results. 

5 Exemplary image analysis, classification, and enhancement/compression 

processes are described herewith in details. Generally, there are three main steps. The 
first step is to analyze an image's histogram, e.g. luminosity histogram, to derive the 
three statistical functions, the Weighted Mean, the Half Value Position, and the 
Weighted Standard Deviation. The second step is to classify the image by its statistical 

1 0 functions which are compared against set parameters for each class of images. The 
classes of images are derived from experimental data. The experimental data can be a 
collection of human experienced/perceptive data about images from which tables and a 
database is built. The third step is to apply a set of predetermined changes to the image 
based upon its class. The changes for each class vary but generally include brightness, 

1 5 contrast, saturation, curve, and unsharp mask. It is appreciated that the changes may be 
applied in a specific degree and order, and that the application of the changes may also 
take into account for the compression that may be applied after this process and/or other 
processes such as resizing of an image. 

I. Analysis of An Input Image 

20 In general, an input image is converted to a standard format, for example, 

a DIB format, to improve visual qualities of the input image. Each image is then 
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analyzed for its unique luminosity and color characteristics/attributes, classified, and 
enhanced to improve its visual qualities. The method includes all interfaces and logic to 
operate in a dynamic environment or a static environment as part of an application or as 
a stand-alone utility. The input image can be a pre-compressed image via a 
5 compression algorithm, such as JPEG, Wavelet, and Fractal, etc. The input image can 
also be an uncompressed image, such as a BMP or TIFF image. 

Analysis of both statistical and visual qualities of the images may 
produce the following results. For example, images can be divided into classes based 
upon distinctions in images' luminosity histogram: a) the Weighted Mean (WM); b) the 

10 Weighted Standard Deviation (WSD); and c) the direction and amount of separation 
between the Weighted Mean and the Half Value Position (HVP), etc. 

By analyzing each component, the following results may occur: a) the 
higher the Weighted Mean, the lighter the image color; it is noted that this may be the 
main determining criteria for image classification; b) the higher the Standard Deviation, 

1 5 the higher the contrast of the image; c) depending on the Weighted Mean and Standard 
Deviation, the difference between the Weighted Mean and the Half Value Position 
determines the exposure of an image. For example, an image with a higher Standard 
Deviation, a greater difference in Half Value Position, and a lesser Weighted Mean, 
results in an over-exposed image. While images that possess these values within a 

20 certain constraint are considered as images of "normal" exposure. This generally forms 
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the basis for the analysis, classification, and identification of the required enhancements 
to be performed on the image. 

The analysis of the image histogram is derived from the three statistical 
functions as defined below: 



255 
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Where p n is defined as the position or ordinal values of the histogram, 
and a n is defined as the amplitude of the pixels at each n th ordinal. 

3) Half Value Position: This is defined as the ordinal in the Histogram where all 
the pixels below that ordinal, sum (left to right) to at least half the pixels of the image. 

It is appreciated that the definitions of the above three functions may be 
varied within the scope of the present invention. 
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II. Classification and Enhancement Application Based on Classification 
The results of the analysis of the image histogram are used to classify 
and enhance the image. 

TABLE 1 below defines a rule set for classifying an image and an 
5 enhancement rule set for each classification of images. It is appreciated that TABLE 1 
is an exemplary illustration of qualitative analysis and can be varied within the scope of 
the present invention. Therefore, TABLE 1 is considered as one embodiment of the 
present invention, and the invention is not limited thereby. 



10 KEYS FOR TABLE 1 



WM= 


Weighted Mean 


B = Brightness 


HV = 


Half Value Position 


C = Contrast 


WSD= 


= Weighted Standard Deviation 


Sat — Saturation 


CRV 


= curves 


Mask = BMP/TIFF Unsharp Mask 



TABLE 1 



Operation Identification 
Shown in Figs. 6A-6C 


Classification Rules 


Enhancement Rules* 


A 


WM<30 


Mask 


Dl 


30<WM<100 

WSD<40 

|WM-HV|<15 


2B, 4C, 2Sat 
20% Mask 


D2 


30<WM<60 

WSD<40 
|WM-HV|>15 


CRV1 
-2B, 2Sat 
20% Mask 


29 
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Operation Identification 
onown in r lgs. oa-oc 


Classification Rules 


Enhancement Rules* 


D3 


60<WM<90 

WSD<40 

I w ivi-n v kij 


2B, AC, 2Sat 
20% Mask 


jj 


90<WM<100 
|WM-HV|>15 


CRV1 
-2B, 2Sat 

OHO/ A/fcioV 

zu /o ivi&sk 


C2 


30<WM<50 
WSD>60 


CRV2 
30% Mask 




50<WM<80 


CRV3 

-JD, "JV^z, J5dl 

30% Mask 


c 


oU* WJVK1UU 
WSD>60 


ir>, -ZL/, j oat 
30% Mask 


b 


30<WM<60 

40<WoD<6U 

|WM-HV|<10 


3B,4C,7Sat 
30% Mask 




60<WM<80 

H\J< W oJJ<t>U 

|WM-HV|<10 


CRV4 

30% Mask 


T 
J 


80<WM<100 

4U<WoJJ<oU 

|WM-HV|<10 


CRV5 
30% Mask 


r 


30<WM<60 

4U<WMJ<ol) 
WM-HV>10 


CRV6 

AT* IC* 

30% Mask 


H 


60<WM<80 

4U<WoJJ<oU 
WM-HV>10 


CRV7 

-zd, j oat 
30% Mask 


K 


80<WM<100 
40<WSD<60 
w ivi-n v i \j 


CRV8 
-2B, 2C 
30% Mask 


I 


60<WM<80 

40<WSD<60 
HV-WM>10 


CRV7 
6B, 5 Sat 
30% Mask 
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V^pciilllUIl JLUvii LlllL.it 11U 11 

Shown in Fias 6A-6C 

0 11 U t t 11 111 1 vf >rv 


v^laaMllCatlull rvtllcs 


11/1111 llllCClllVll l XYUlCiS 


L 


80<WM<100 
40<WSD<60 
HV-WM>10 


CRV7 

6B 2C 5Sat 
30% Mask 


A4 


WM>220 


30% Mask 


Al 


WSD>60 


J U /0 ivlabiv 


A9 


200<WM<220 
WSD<40 


3C ? lOSat 

90% IVT^qV 
ZU/0 IvlaoK 




200<WM<220 
40<WSD<60 


CRV8 

Ju/O lvla£>Jv 


X 


150<WM<175 
WSD>60 

| WlVl-xl V j <1 J 


IB, -3C, 3Sat 
30% Mask 


VI 

Al 


175<WM<200 

Wo.U>oU 
|WM-HV|<15 


CRV9 
2C, 3Sat 

JU/o iviasK 


w 


150<WM<175 
|WM-HV|>15 


CRV10 

jD, Zl^ ? OOal 

30% Mask 


Wl 


1 / WIVISZUU 

WSD>60 
|WM-HV|>15 


^Iv V 1U 

4B, 4Sat 
30% Mask 


v 
I 


150<WM<200 

WiSJLi<4U 


-IB, 4C 

jU/o IVLaSK 


Z2 


150<WM<200 
40<WSD<60 

I W Ivl-li V I I J 


CRVll 
2C, 5 Sat 
30% Mask 


z 


150<WM<175 

40<WSD<60 
|WM-HV|<15 


2C, 4Sat 
22% Mask 


Zl 


175<WM<200 

40<WSD<60 
|WM-HV|<15 
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-2B, 3 Sat 
30% Mask 
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V7pcl illlUIl lUeillllHallUIl 

Shown in Figs. 6A-6C 


V^laaMlIVrtllUU JCVUlCa 


TTn li tjnppin ph t Riilp^^ 


1VJL 


100<WM<120 
45<WSD< < i5 

'T.J-i: VY OL'-iJ J 

30% Mask 


CRV12 
7R S Sat 
30% Mask 


N 


i?o<WM<no 

I Zs\J ^ VV lvl_i JL .J v 

45<WSD<55 


CRV12 
IB, 3 Sat 
30% Mask 




130<WM<150 


30% Mask 


p 


100<WM<150 

VV uL/ x TU 


CRV13 
-4B, -3 Sat 
?0% Mask 


Q 


100<WM<1 SO 

40<WSD<45 


JV) JOCIL 

20% Mask 


Rl 


55<WSD<60 


jU/O lVIaoiv 


XX 


130<WM<150 

■ VV 0 1 J ::A}\! 


CRV12 
30% Mask 


V 1 


100<WM<130 
|WM-HV|<8 


CRV14 
-1R 4Sat 

30% Mask 


V 


130<WM<150 
w^tv>70 

W o D-^ 1 U 

|WM-HV|<8 


CRV15 

~JV_^, / DcLL 

30% Mask 


si 


100<WM<130 
WSD>60 

1 w lvi-n v | ^ u 


CRV16 
-2B, 1C, 2Sat 


c 
o 


130<WM<150 
|WM-HV|<15 


CRV17 
IB, -3Sat 

"30% Mask 

J \J /0 IVldoJS. 


T 


100<WM<130 

WSD>60 

WM-HV>15 


CRV18 
1C, 6Sat 
30% Mask 


Tl 


130<WM<150 

WSD>60 

HV-WM>15 
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-IB, 2Sat 
30% Mask 
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OtiPfaf inn Tflpfitifipafinn 


tf ni liiccifip$i'HnTi ul pc 


JLIilldllCclIlclll JVUlcs 


OI1UWI1 111 r vr\"UV/ 








1 UUS W IVI^ 1 J u 


pt? von 


u 


WSD>60 


-4B 5 Sat 




HV-WM>15 


30% Mask 




130<WM<150 


CRV21 


Ul 


WSD>60 


-2B, 2Sat 




WM-HV>15 


30% Mask 



Figs. 6A-6C illustrate a flow diagram of image classification/ 
enhancement operations in one embodiment of an image processing method in 
accordance with the principles of the present invention. The image enhancement 
5 operations or rules shown in Figs. 6A-6C correspond to TABLE 1 . For example, an 
image having the statistics in TABLE 1 A below would apply image enhancement rule 
"G" in Fig. 6A ? following operational blocks 204-222. The order in which the 
enhancement functions are specified in the rule set is one implementation of present 
invention. It is appreciated that the other orders can be used without departing from the 
1 0 present invention. 



TABLE 1A 



Weighted Mean 


Weighted Standard Deviation 


Half Value Position 


66.3 


45.9 


72 



It is appreciated that other classification/enhancement rules can be used 
without departing from the scope of the invention. It is also appreciated that the above 
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classification/enhancement rules may be implemented in an algorithm format within the 
scope of the invention. 

It is appreciated that the other image processing operations such as 
sharpening, masking, etc., can be used within the scope of the present invention. It is 
5 also appreciated that the degree of adjustment in brightness, contrast, saturation, or the 
percentage or amount of curve, unsharp mask, etc., can be varied without departing 
from the scope of the present invention. 

Exemplary image processing filters or tools that can be used in the 
present invention are Brightness, Contrast, Saturation, Curves, Unsharp Mark, Preserve 
1 0 White, Resizing, etc. 

It is appreciated that the image processing filters are not limited to the 
above, and that the other filters can be used, such as color balance, tone balance, and/or 
hue, cropping, etc. 

Figs. 7A-7C illustrate exemplary curves. As described above, curves 
1 5 allow enhancement of luminosity characteristics in a given area of an image when 

determined by the classification process. The curves enhancement function is defined as 
a re-mapping of luminosity or density values versus exposure. The benefit of this 
function occurs when an image needs a shift in luminosity for specific portions in the 
image. For example, images with shadows are able to shift those shadows to a higher 
20 luminosity while the entire image is not shifted or made lighter. The way of achieving 
this technique involves a mapping of luminosity values which are valued from 0-255 to 
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new luminosity values in that range. The curve applied to an image is determined by 
the classification of the image. Three exemplary curves applied to map the luminosity 
values of an input image are shown in Figs. 7A-7C, respectively. In Figs. 7A-7C, the 
line starts in the lower left, i.e. at black color (luminosity level of 0) and goes to the 
5 upper right, i.e. at white color (luminosity level of 255). In Fig. 7 A, a curve 224 

demonstrates no alteration or mapping to an image. For example, an input luminosity 
level of 143 remains at 143. In Fig. 7B, a curve 226 is applied to brighten the entire 
image. For example, an input luminosity level of 143 is brightened to luminosity level 
of 184. In Fig. 7C, a curve 228 is applied to darken the over-all image. For example, an 

1 0 input luminosity level of 143 is darkened to luminosity level of 109. It is noted that in 
Figs. 7B and 7C, an entire image is brightened or darkened. It is appreciated that a 
curve can be applied to brighten or darken a certain portion of the image. It is also 
appreciated that other curves can be used within the scope of the invention. 

To further illustrate a curve in a table format, TABLE 2 provides one 

1 5 exemplary curve for luminosity mapping. 

TABLE 2: Curves Table Example 



Curves Table Example 



255 


254 


253 


252 


251 


250 


249 


248 


247 


255 


253 


251 


249 


248 


246 


244 


242 


241 




















246 


245 


244 


243 


242 


241 


240 


239 


238 


239 


237 


235 


234 


232 


231 


229 


228 


226 




















237 


236 


235 


234 


233 


232 


231 


230 


229 


225 


223 


222 


221 


220 


219 


218 


217 


216 
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Curves Table Example 



228 


227 


226 


225 


224 


223 


222 


221 


220 


215 


214 


213 


212 


211 


210 


209 


208 


207 
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191 


190 


189 


189 


188 


187 


186 


186 




















183 


182 


181 


180 


179 


178 


177 


176 


175 


185 


184 


183 


183 


182 


181 


180 


180 


179 




















174 


173 

J. / ~j 


172 


171 


170 


169 


168 


167 


166 


178 

I/O 


111 


176 


175 


174 


173 


172 


171 


170 




















165 


164 


163 


162 


161 


160 


159 


158 


157 


169 


168 

X \J\J 


167 


166 


165 


164 


163 


162 


162 




















156 


155 


154 


153 


152 


151 


150 


149 


148 


160 


159 


158 


157 


156 


155 


154 


153 


152 




















147 


146 


145 


144 


143 


142 


141 


140 


139 




149 


148 


147 


146 


145 


144 


143 


142 






















117 


136 


1 J u 


134 


133 

JL 3 3 


132 


131 

X JL 


130 


141 


140 


138 

X ~J KJ 


137 


136 


135 


134 


133 


132 




















129 


128 


127 


126 


125 


124 


123 


122 


121 


131 


130 


129 


127 


126 


125 


124 


123 


122 




















120 


119 


118 


117 


116 


115 


114 


113 


112 


121 


120 


119 


117 


116 


115 


114 


113 


112 





















36 

Attorney Docket No. 13095.1-US-U1 



Curves Table Example 
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Further, an unsharp mask enhances an image by making it sharper 
without accentuating small imperfections in an image. Thus, an image gains more 
details and clarity while preserving visual continuity. The unsharp mask percentage 
applied to an image is dependent upon an input image file type and is handled 
5 differently for a compressed image, such as a JPEG image, from an uncompressed 
image, such as a BMP or TIFF image. 

Pre-compression Enhancement Process for Images that are Resized 

The image processing system in accordance with the present invention 
addresses issues of image quality degradation resulting from either resizing of an image 
10 or artifacts introduced in the process of compressing a previously compressed image, or 
a combination of both processes. 

Post Resizing Enhancement Process 

High quality digital source images, such as BMP and TIFF files that have 
not been previously compressed, may need to be resized. If the image is resized by 

1 5 dividing the pixel dimensions by some number that, when divided into the number of 
pixels leaves no fraction, then the sharpness of the image is not significantly degraded 
in the process. For example, reducing 400 pixels to 200 pixels or 900 pixels to 300 
pixels would not greatly alter the apparent sharpness of the image. If the image is 
resized by another factor such as 400 pixels to 299 pixels, significant interpolation of 

20 the image occurs resulting in the loss of image sharpness or clarity. The loss of 

sharpness generated from this interpolation process can be significantly regained by 
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increasing the edge definition of the image content through application of an unsharp 
mask filter. The percentage strength of the mask can be altered to optimize the amount 
of edge enhancement applied based on the ratio of reduction from the original image, 
the absolute physical dimensions of the reduced image, the amount of interpolation 
5 required to achieve the desired resized dimensions, the type of interpolation algorithm 
used, or a combination of all of the above factors. 

TABLE 3 below defines an exemplary unsharp mask applied to a resized 
image, which is not previously compressed, based on the image's longest axis 
dimension. 

10 

TABLE 3 (BMP & TIFF unsharp mask) 



MASK TABLE FOR BMP & TIFF IMAGES 


DPI 


Mask % to be applied 


0 to 128 


30 


128 to 450 


35 


450+ 


30 



1 5 The enhancement process also considers the special case of thumbnails, 

e.g. images of approximately 128 pixels or less on the longest axis, wherein the 
information defining the image is significantly reduced. Image sharpness of thumbnail- 
sized images is greatly improved by applying an additional percentage of unsharp mask, 
and a small increase in contrast, brightness, and, for color images, saturation. These 
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filters allow the image to retain high visual qualities while attaining higher degrees of 
compression than when not applying such filters. 

It is appreciated that the enhancement process is not limited to the special 
case of thumbnails, and that the enhancement process can be used in other resizing 
applications within the scope of the present invention. 

Enhancement Process for Resizing Previously Compressed Images 
The rules that govern usage of an unsharp mask applied to an input 
image previously compressed, such as a JPEG image, require different unsharp mask 
optimization rules from those applied to images that have not been previously 
compressed. 

The percent of unsharp mask to be applied in the resizing of a previously 
compressed JPEG image requires compensating for the variables described as optimal 
for uncompressed images and, in most cases, the percentage of the unsharp mask is then 
reduced from that uncompressed optimal setting so as not to accentuate the artifacts 
introduced in the initial compression process. 

Determining the factor to reduce the amount of mask for a given JPEG 
file can be calculated in several ways. In one method, the ratio of total pixel count of 
the DIB (Device Independent Bitmap) is compared with the original compressed JPEG 
image file size in kilobytes. This ratio is used as criteria for exemplary TABLE 4 below 
to obtain the appropriate unsharp mask percentage to be applied to the image. 
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TABLE 4 



MASK TABLE FOR JPEG IMAGES 



Ratio is the total pixels of an image divided by the file 

size (K) 



Ratio 


Mask % to be applied 


0-1500 


42 


1501-1900 


36 


1901-2300 


26 


2301-2600 


23 


2601-3500 


20 


3501-4500 


17 


4501-6000 


15 


6001-9000 


10 


9001-12000 


7 


12001+ 


5 



The application of the use of an unsharp mask in this fashion enhances 
edge definition and, therefore, apparent sharpness to the image without adding 
significantly to the picture's high frequency content. Since high frequency limits how 
effectively a compression algorithm can be in reducing file size, images processed in 
this manner can be compressed further, with less visible artifacts, high apparent image 
sharpness, and greatly reduced file size. 
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The pre-compression, post resizing enhancement is applied automatically 
without operator's intervention. The algorithm can also be applied in a networked or 
server-based environment where no human intervention is possible. 



5 Enhancement Process for Reducing Apparent Artifacts in Images 

An image may have had artifacts introduced into the picture or image 
through a scanning process, compression algorithm, color reduction such as experienced 
in a GIF image for the Internet or by scanning a printed page including half-tone dot 
patterns. The present invention reduces these visible artifacts and allows for the 
10 creation of a second-generation image that can be resized and compressed with 

improved edge definition and reduced artifacts as compared with the source image. 

The process utilizes a "Blur" filter that tends to "smear/smudge" the 
blocking generated as a result of the compression or similar to a blocking of dot patterns 
created by scanned images or GIF. Because the "Blur" filter reduces the appearance of 
1 5 the blocking throughout the whole picture, edge definition of the image degrades and 
thus must be restored. The application of unsharp mask filter is then applied in multiple 
iterations in same or varying amounts (e.g. 15%/15%/15%) whereupon the image's 
intended edges are redefined with the artifacts remaining less evident. 

It is appreciated that other filters may be applied to pre-compression, 
20 post-resizing enhancing of images without departing from the scope of the present 
invention. It is further appreciated that the degree of application of the filters applied, 
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e.g. "Blur", "unsharp mask", etc., can be varied without departing from the scope of the 
present invention. 

III. Compression of the Output Image 

The image processing system may include a user interface that allows a 
5 user to compress an enhanced image. As an example, an enhanced JPEG image can be 
further compressed by setting YUV settings and Q-factor, as well as a unique file name 
for each output image when applied to a batch of images. 

It is appreciated to those skilled in the art that the Q-Factor relates to a 
term that represents the quality versus compression ratio used in saving an image as a 
10 JPEG file. It is also appreciated to those skilled in the art that a YUV Color Difference 
Coordinate System relates to a color coordinate system that represents RGB colors in 
terms of the luminosity Y and two difference colors. The Y tristimulus value represents 
luminosity, while the U and V variables represent a percentage of luminosity less Blue 
and Red, respectively. It is appreciated that the compression applied is not limited to 
15 the compressed JPEG image, without departing from the scope of the present invention. 

IV. Compression of an Image to a Target File Size 

The image processing system includes a user interface that allows a user 
to drive an input image to an approximate file size. In one embodiment of the present 
20 invention, this may be accomplished with a series of tables. This feature involves 

saving an input image in a memory or buffer as a 30 Q-factor/4:4:4 YUV setting. The 
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file size (in kilobytes) is then compared against a file size, such as a BMP file size, of an 
input image to obtain a percentage. This percentage is referenced and used to map to 
the correct table. An example of these tables is given in a JPEG Compression 
Approximation in TABLE 5. Of course, those skilled in the art appreciate that many 
5 examples of such tables may be provided, and that TABLE 5 is but one of such 

examples. Accordingly, the invention is not limited to such examples as others may be 
provided without departing from the scope and spirit of the invention. 

TABLE 5 



YUV 
Setting 


QFactor j~ 


;<JFactor 1 


QFactor 
10 


jjjjQFactor 
15 


. QFaeiorll 


QFactor jj QFactor 


QFactor 
35 


i: # | 


411 


, 2.47 


1.98 


1.65 


1.36 


1 12 


1.07 


I 1 


0 


: 0.856 


f 422 


""T695"" ? 


2.075 ; 


1.675 


'""*L38 ~" 


: : 1.21 


"T675 


'f 1 


0.925 


'"1)"867" " ; 


: 444 




2J7 ' ' 




1.4 


"~\22 " : 


1.08 


V"'' i "' 




"¥878 ; 



10 



An exemplary process of compressing an image on a single image input 
basis is as follows: 

1 . Saving the image (enhanced or not) as a 30 Q factor JPEG, 444 
1 5 JPEG to a memory or buffer, recording the file size of this image, and 

using it as a reference. 

2. Multiplying this file size by the factors dependent upon the YUV 
setting chosen by the user. 
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3. Depending upon the file size chosen, the appropriate Q-Factor is 
applied to the image in order to output the image as a JPEG image. 



Accordingly, the user interface allows for various compression and/or 
5 quality change settings. Any known compression algorithm, such as the JPEG 
compression algorithm, can be used within the scope of the invention. In one 
embodiment, the first user option is a series of compression choices that involve either a 
slider bar or a set of numerical options. The user interface allows for image 
compression ranging from, for example, 20 to 60 in Q-Factor while utilizing a 4: 1 : 1 
10 YUV setting. 

The image compression range, for example, from 20 to 60 in Q-Factor 
while utilizing a 4:1:1 YUV setting, is imposed to limit the degradation of images that 
normally occurs beyond a 60 Q-Factor. For example, a 4:1 :1 YUV setting causes a 
need for an additional +5 saturation to compensate for the color change in an image. 

1 5 The user interface also allows for a separate quality choice. This gives the user the 

option to compress an image while still being assured a high quality image. In another 
example, the Q-Factor ranges from 2 to 30 and involves a YUV setting of 4:4:4, which 
may not require any additional saturation. 

The above specification, examples and data provide a complete and 

20 enabling description of the invention. Since many embodiments of the invention can be 
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made without departing from the spirit and scope of the invention, the invention resides 
in the claims hereinafter appended. 
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CLAIMS 

What is claimed is: 

1 . A method of automatically processing a series of input images without operator's 
intervention, in automatically processing each of the input images, the method 

5 comprising: 

providing processing parameters; 

automatically analyzing, classifying, and enhancing the input image without 
operator's intervention; and 

outputting at least one enhanced image. 

10 

2. The method of claim 1, further comprising outputting a plurality of enhanced 
images with different file types without re-analyzing, re-classifying, and re-enhancing 
the input image. 

15 3 . The method of claim 1 , further comprising outputting a plurality of enhanced 
images with different file sizes without re-analyzing, re-classifying, and re-enhancing 
the input image. 

4. The method of claim 1 , wherein the input images are processed in real time on- 
20 the-fly. 
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5. 



The method of claim 1, wherein the input images are processed in a batch mode. 



6. The method of claim 1, wherein the input images are processed in an embedded 
software. 

5 

7. The method of claim 1, wherein the input images are processed in an embedded 
hardware. 

8. The method of claim 1, wherein the input images are processed in an embedded 
10 firmware. 

9. The method of claim 1, wherein the step of automatically analyzing, classifying, 
and enhancing the input image includes applying an individual image enhancement 
independently to each of the input images. 

15 

1 0. The method of claim 1 , further comprising pre-processing the input image 
before analyzing, classifying, and enhancing the input image. 

1 1 . The method of claim 1 , further comprising resizing the enhanced image. 

20 

12. The method of claim 1, further comprising compressing the enhanced image. 
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1 3 . The method of claim 1 1, further comprising compressing the enhanced image 
before outputting the resized image. 

5 14. The method of claim 13, further comprising automatically applying at least one 
filter in a pre-compression, post-resizing enhancement. 

15. The method of claim 13, further comprising post-processing the enhanced image 
before compressing the enhanced image. 

10 

16. The method of claim 1, wherein the processing parameters define the number of 
the output images. 

17. The method of claim 1 1 , wherein the processing parameters define resizing 
15 parameters. 

1 8. The method of claim 12, wherein the processing parameters define compression 
parameters. 

20 19. The method of claim 1 3 , wherein the processing parameters define compression 
parameters. 
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20. The method of claim 1, further comprising viewing the enhanced image before 



outputting. 

5 21 . The method of claim 1, wherein the input images are processed in a server 

computer of a network system, an output destination for the output image is coupled to 
the network system. 

22. The method of claim 21, wherein the network system is Internet, whereby the 
1 0 system is capable of being applied in an automated on-the-fly environment where 

images are sent to the server computer via the Internet whereupon the images are 
processed and transmitted to the output destination. 

23 . A method of automatically processing a series of input images without operator's 
1 5 intervention, in automatically processing each of the input images, the method 

comprising: 

providing processing parameters; 

automatically analyzing and classifying each of the input images without 
operator's intervention; 
20 automatically compressing the image; and 

outputting at least one compressed image. 
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24. The method of claim 23, further comprising outputting a plurality of compressed 
images with different file types without re-analyzing and re-classifying each of the 
images. 

5 

25. The method of claim 23, further comprising outputting a plurality of compressed 
images with different file sizes without re-analyzing and re-classifying the input image. 

26. The method of claim 23, wherein the input images are processed in real time on- 
10 the-fly. 

27. The method of claim 23, wherein the input images are processed in a batch 
mode. 

15 28. The method of claim 23, wherein the input images are processed in an embedded 
software. 

29. The method of claim 23, wherein the input images are processed in an embedded 
hardware. 

20 

30. The method of claim 23, wherein the input images are processed in an embedded 
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firmware. 



3 1 . The method of claim 23 , wherein the step of automatically analyzing, 
classifying, and enhancing the input image includes applying an individual image 

5 enhancement independently to each of the input images. 

32. The method of claim 23, further comprising pre-processing the input image. 

33. The method of claim 23, further comprising resizing the classified image before 
1 0 compressing the image. 

34. The method of claim 33, further comprising automatically applying at least one 
filter in a pre-compression, post-resizing enhancement. 

15 35. The method of claim 34, further comprising post-processing the resized image 
before compressing the image. 

36. The method of claim 23, wherein the processing parameters define the number 
of the output images. 

20 

37. The method of claim 33, wherein the processing parameters define resizing 
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parameters. 



38. The method of claim 23, wherein the processing parameters define compression 
parameters. 

5 

39. The method of claim 23 , further comprising viewing the compressed image 
before outputting. 

40. The method of claim 23 , wherein the input images are processed in a server 

1 0 computer of a network system, an output destination for the output image is coupled to 
the network system. 

41 . The method of claim 40, wherein the network system is Internet, whereby the 
system is capable of being applied in an automated on-the-fly environment where 

15 images are sent to the server computer via the Internet whereupon the images are 
processed and transmitted to the output destination. 

42. An image processing system for automatically processing a series of input 
images without operator's intervention, comprising: 

20 an image analysis module for automatically analyzing each of the input images; 

an image classification module for automatically classifying the analyzed image; 
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an image enhancement module for automatically enhancing the classified image 
based on classification; and 

an output module for outputting at least one enhanced image. 

5 43. The system of claim 42, wherein the output module outputs a plurality of 

enhanced images with different file types without re-analyzing, re-classifying, and re- 
enhancing the input image. 

44. The system of claim 42, wherein the output module outputs a plurality of 

10 enhanced images with different file sizes without re-analyzing, re-classifying, and re- 
enhancing the input image. 

45. The system of claim 42, wherein the image analysis module, the image 
classification module, the image enhancement module, and the output module process 

1 5 the input image in real time on-the-fly . 

46. The system of claim 42, wherein the image analysis module, the image 
classification module, the image enhancement module, and the output module process 
the input image in a batch mode. 

20 

47. The system of claim 42, wherein the image analysis module, the image 



54 

Attorney Docket No. 13095. 1-US-U1 



classification module, the image enhancement module, and the output module process 
the input image in an embedded software. 



48. The system of claim 42, wherein the image analysis module, the image 

5 classification module, the image enhancement module, and the output module process 
the input image in an embedded hardware. 

49. The system of claim 42, wherein the image analysis module, the image 
classification module, the image enhancement module, and the output module process 

10 the input image in an embedded firmware. 

50. The system of claim 42, wherein the image enhancement module applies an 
individual image enhancement independently to each of the input images. 

15 51. The system of claim 42, further comprising a pre-processor for pre-processing 
the image before analyzing, classifying, and enhancing the input image. 

52. The system of claim 42, further comprising an image resizer for resizing the 
enhanced image. 

20 

53 . The system of claim 42, further comprising an image compression module for 
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compressing the enhanced image before outputting the enhanced image. 



54. The system of claim 52, further comprising an image compression module for 
5 compressing the enhanced image before outputting the resized image. 

55. The system of claim 54, further comprising a pre-compression post-resizing 
image enhancement module for automatically applying at least one filter before 
compressing the enhanced image. 

10 

56. The system of claim 54, further comprising a post-processor for post-processing 
the enhanced image before compressing the enhanced image. 



57. The system of claim 42, further comprising an input module for inputting the 
1 5 input image and a set of parameters, the input module is capable of inputting multiple 
system, user defined naming conventions for the output image as the set of parameters. 



58. The system of claim 57, wherein the parameters define the number of the output 
images. 

20 

59. The system of claim 57, wherein the parameters define resizing parameters. 
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60. The system of claim 57, wherein the parameters define compression parameters. 

61 . The system of claim 42, further comprising a viewer for viewing the enhanced 
5 image before outputting. 

62. The system of claim 42, wherein the input images are processed in a server 
computer of a network system, an output destination for the output image is coupled to 
the network system. 

10 

63. The system of claim 62, wherein the network system is Internet, whereby the 
system is capable of being applied in an automated on-the-fly environment where 
images are sent to the server computer via the Internet whereupon the images are 
processed and transmitted to the output destination. 

15 

64. The system of claim 42, wherein the image analysis module analyzes the image 
based on an image histogram. 

65. The system of claim 42, wherein the image classification module classifies the 
20 image based on a Weighted Means range, a Weighted Standard Deviation range, and a 

Halve Value Position range of an image luminosity histogram. 
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66. The system of claim 42, wherein the image enhancement module includes a 
brightness filter. 

5 67. The system of claim 42, wherein the image enhancement module includes a 
contrast filter. 

68. The system of claim 42, wherein the image enhancement module includes a 
saturation filter. 

10 

69. The system of claim 42, wherein the image enhancement module includes a 
curve filter. 

70. The system of claim 42, wherein the image enhancement module includes an 
1 5 unsharp mask filter. 

71 . The system of claim 53, wherein the image compression module includes a 
JPEG compression. 

20 72. The system of claim 52, wherein the image resizer resizes the image to a 
thumbnail size. 
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73. 



The system of claim 51, wherein the image pre-processor includes a brightness 



filter. 

5 74. The system of claim 5 1 , wherein the image pre-processor includes a contrast 
filter. 

75. The system of claim 5 1 , wherein the image pre-processor includes a saturation 
filter. 

10 

76. The system of claim 5 1 , wherein the image pre-processor includes an unsharp 
mask filter. 

77. The system of claim 56, wherein the image post-processor includes a brightness 
15 filter. 

78. The system of claim 56, wherein the image post-processor includes a contrast 
filter. 

20 79. The system of claim 56, wherein the image post-processor includes a saturation 
filter. 
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80. The system of claim 56, wherein the image post-processor includes an unsharp 
mask filter. 

5 81. The system of claim 55, wherein the pre-compression post-resizing image 
enhancement module includes an unsharp mask filter. 

82. An image processing system for automatically processing a series of input 
images without operator's intervention, comprising: 

10 an image analysis module for automatically analyzing each of the input images; 

an image classification module for automatically classifying the analyzed image; 
an image compression module for automatically compressing the classified 
image based on classification; and 

an output module for outputting at least one compressed image. 

15 

83. The system of claim 82, wherein the output module outputs a plurality of 
enhanced images with different file types without re-analyzing and re-classifying the 
input image. 

20 84. The system of claim 82, wherein the output module outputs a plurality of 
enhanced images with different file sizes without re-analyzing and re-classifying the 
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input image. 

85. The system of claim 82, wherein the image analysis module, the image 
classification module, the image compression module, and the output module process 

5 the input image in real time on-the-fly. 

86. The system of claim 82, wherein the image analysis module, the image 
classification module, the image compression module, and the output module process 
the input image in a batch mode. 

10 

87. The system of claim 82, wherein the image analysis module, the image 
classification module, the image compression module, and the output module process 
the input image in an embedded software. 

15 88. The system of claim 82, wherein the image analysis module, the image 

classification module, the image compression module, and the output module process 
the input image in an embedded hardware. 

89. The system of claim 82, wherein the image analysis module, the image 
20 classification module, the image compression module, and the output module process 
the input image in an embedded firmware. 
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90. The system of claim 82, wherein the image analysis module, the image 
classification module, the image enhancement module, and the output module apply an 
individual image enhancement independently to each of the input images. 

91 . The system of claim 82, farther comprising a pre-processor for pre-processing 
the input image before analyzing and classifying the input image. 



10 92. The system of claim 82, further comprising an image resizer for resizing the 
enhanced image before outputting the compressed image. 

93. The system of claim 91, further comprising a pre-compression post-resizing 
image enhancement module for automatically applying at least one filter before 

1 5 compressing the enhanced image. 

94. The system of claim 92, further comprising a post-processor for post-processing 
the input image before compressing the enhanced image. 

20 95 . The system of claim 82, further comprising an input module for inputting the 
input image and a set of parameters, the input module is capable of inputting multiple 
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system, user defined naming conventions for output images as the set of parameters. 

96. The system of claim 95, wherein the parameters define the number of the output 
images. 

5 

97. The system of claim 95, wherein the parameters define resizing parameters. 

98. The system of claim 95, wherein the parameters define compression parameters. 

10 99. The system of claim 82, further comprising a viewer for viewing the compressed 
image before outputting. 

100. The method of claim 82, wherein the input images are processed in a server 
computer of a network system, an output destination for the output image is coupled to 

1 5 the network system. 

101. The method of claim 1 00, wherein the network system is Internet, whereby the 
system is capable of being applied in an automated on-the-fly environment where 
images are sent to the server computer via the Internet whereupon the images are 

20 processed and transmitted to the output destination. 
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102. The system of claim 82, wherein the image analysis module analyzes the input 
image based on an image histogram. 

1 03. The system of claim 82, wherein the image classification module classifies the 
input image based on a Weighted Means range, a Weighted Standard Deviation range, 
and a Halve Value Position range of an image luminosity histogram. 

104. The system of claim 82, wherein the image compression module includes a 
JPEG compression. 

1 05. The system of claim 92, wherein the image resizer resizes the image to a 
thumbnail size. 

1 06. The system of claim 91 , wherein the image pre-processor includes a brightness 
filter. 

107. The system of claim 91, wherein the image pre-processor includes a contrast 
filter. 

1 08. The system of claim 91 , wherein the image pre-processor includes a saturation 
filter. 
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1 09. The system of claim 91 , wherein the image pre-processor includes an unsharp 
mask filter. 

1 10. The system of claim 94, wherein the image post-processor includes a brightness 
filter. 

111. The system of claim 94, wherein the image post-processor includes a contrast 
filter. 

112. The system of claim 94, wherein the image post-processor includes a saturation 
filter. 

113. The system of claim 94, wherein the image post-processor includes an unsharp 
mask filter. 

1 14. The system of claim 93, wherein the pre-compression post-resizing image 
enhancement module includes an unsharp mask filter. 

115. A computer program storage medium readable by a computing system and 
encoding a computer program of instructions for executing a computer process of 
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automatically processing a series of input images without operator's intervention, in 
automatically processing each of the input images, the computer process comprising: 
providing processing parameters; 

automatically analyzing, classifying, and enhancing the input image without 
5 operator's intervention; and 

outputting at least one enhanced image. 

116. A computer data signal embodied in a carrier wave readable by a computing 
system and encoding a computer program of instructions for executing a computer 

10 process of automatically processing a series of input images without operator's 

intervention, in automatically processing each of the input images, the computer process 
comprising: 

providing processing parameters; 

automatically analyzing, classifying, and enhancing the input image without 
1 5 operator's intervention; and 

outputting at least one enhanced image. 

117. A computer program storage medium readable by a computing system and 
encoding a computer program of instructions for executing a computer process of 

20 automatically processing a series of input images without operator's intervention, in 
automatically processing each of the input images, the computer process comprising: 
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providing processing parameters; 

automatically analyzing and classifying the input image without operator's 
intervention; 

automatically compressing the image; and 
5 outputting at least one compressed image. 



118. A computer data signal embodied in a carrier wave readable by a computing 
system and encoding a computer program of instructions for executing a computer 
process of automatically processing a series of input images without operator's 
10 intervention, in automatically processing each of the input images, the computer process 
comprising: 

providing processing parameters; 

automatically analyzing and classifying the input image without operator's 
intervention; 

15 automatically compressing the image; and 

outputting at least one compressed image. 
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ABSTRACT 

Image processing system and method automatically process a series of 
input images without operator's intervention. Each of the input images is automatically 
analyzed, classified, enhanced and/or compressed. Individual image enhancement is 

5 applied to each of the input images independently. A plurality of enhanced and/or 
compressed images can be outputted without re-analyzing and re-classifying the input 
image. The processed images have a high compression to quality ratio, produce images 
having quality and visual characteristics as viewed by a human including image 
contrast, brightness, and sharpness as compared with the input image, while capable of 

10 reducing the amount of memory required for storing the output images. The image 
processing system and method can process the images in real time on-the-fly or in a 
batch mode, and can operate in a wire and/or wireless transmission environment or 
embedded in a hardware, firmware, or software which is embedded in another system or 
application. The system can be a stand-alone system or a server/client network system, 

15 etc. 
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represented unless/until I instruct Merchant & Gould P.C. to the contrary. 

Please direct all correspondence in this case to Merchant & Gould P.C. at the address indicated below: 

Merchant & Gould P.C. 
P.O. Box 2903 
Minneapolis, MN 55402-0903 




2 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and belief are 
believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false statements 
may jeopardize the validity of the application or any patent issued thereon. 
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§ 1.56 Duty to disclose information material to patentability. 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, and the most effective 
patent examination occurs when, at the time an application is being examined, the Office is aware of and evaluates the teachings of all 
information material to patentability. Each individual associated with the filing and prosecution of a patent application has a duty of candor 
and good faith in dealing with the Office, which includes a duty to disclose to the Office all information known to that individual to be 
material to patentability as defined in this section. The duty to disclose information exists with respect to each pending claim until the 
claim is canceled or withdrawn from consideration, or the application becomes abandoned. Information material to the patentability of a 
claim that is canceled or withdrawn from consideration need not be submitted if the information is not material to the patentability of any 
claim remaining under consideration in the application. There is no duty to submit information which is not material to the patentability of 
any existing claim. The duty to disclose all information known to be material to patentability is deemed to be satisfied if all information 
known to be material to patentability of any claim issued in a patent was cited by the Office or submitted to the Office in the manner 
prescribed by §§ L97(b)-(d) and 1.98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. The Office encourages 
applicants to carefully examine: 

(1) prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) the closest information over which individuals associated with the filing or prosecution of a patent application 
believe any pending claim patentably defines, to make sure that any material information contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to information already of record or 
being made of record in the application, and 

O (1) It establishes, by itself or in combination with other information, a prima facie case of unpatentability of a claim; 

or -j 

H (2) It refutes, or is inconsistent with, a position the applicant takes in: 

:V ~ (i) Opposing an argument of unpatentability relied on by the Office, or 

: p (ii) Asserting an argument of patentability. 

A p : Mma facie case of unpatentability is established when the information compels a conclusion that a claim is unpatentable under the 
prefy&nderance of evidence, burden-of-proof standard, giving each term in the claim its broadest reasonable construction consistent with the 
specification, and before any consideration is given to evidence which may be submitted in an attempt to establish a contrary conclusion of 
patentability, 

S (c) Individuals associated with the filing or prosecution of a patent application within the meaning of this section are: 

(1) Each inventor named in the application: 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the application and who is 
associated with the inventor, with the assignee or with anyone to whom there is an obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by disclosing information to the 
attorney, agent, or inventor. 
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